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Abstract 

We update phenomenological constraints on a Two Higgs Doublet Model with lepton flavour non-conserving 
Yukawa couplings. We review that tan/3 is ambiguous in such "Type III" models, and define it from 
the t Yukawa coupling. The neutral scalars 4> could be searched for at hadron colliders in <j> -» rp,, and 
are constrained by the rare decay r — >• fiy. The Feynman diagrams for the collider process, with Higgs 
production via gluon fusion, are similar to the two-loop "Barr-Zee" diagrams which contribute to t -> fj,-y. 
Some "tuning" is required to obtain a collider cross-section of order the Standard Model expectation for 
°"(.9.9 — * hsM — > t + t~), while agreeing with the current bound from r —> fij. 



1 Introduction 

The Two Higgs Doublet Model (2HDM) may be the low energy effective theory for many models of Beyond-the 
Standard Model (BSM) physics at the TeV scale. Various Higgses could be the first signals of BSM physics 
discovered at hadron colliders. The aim of this paper is therefore to study the implications for collider searches, 
of precision physics bounds on a generic 2HDM with lepton flavour violating couplings. Similar analyses have 
previously been performed in [1, 2, 3, 4, 5, 6]. We assume that the additional Higgses are the only BSM particles 
with masses < 400 GeV, and consider constraints on the Higgs parameters which are comparatively independent 
of additional New Physics at higher scales. 

The 2HDM [7] (see [8] for an introduction) consists in adding a second Higgs doublet to the Standard Model. 
Despite being a fairly minimal extension 1 , it has many variants. In particular, a discrete symmetry [12] can 
be imposed on the Higgs plus fermion Lagrangian, to avoid tree level flavour changing neutral interactions. 
We focus here on "Type III" models, meaning that no discrete symmetries are present, so there is no unique 
definition of tan /3. We neglect non-renormalisable operators [13, 14] in the potential. In the absence of a discrete 
symmetry, the fermions can couple to both Higgs doublets with generic Yukawa matrices, which allows flavour- 
changing tree-level couplings of the physical Higgses. For simplicity, we make the (unrealistic) assumption that 
our Higgses only have lepton flavour violating interactions 2 . We are particularily interested in the Higgs— r — /i 
interaction. 

We emphasize that, for a generic neutral Higgs <fi, the search for (j> — > r/2 at hadron colliders[16, 3, 4] should 
take into account the bounds on r — > /^7[5]. In figure 1, on the left is shown a fermion loop diagram of "Barr- 
Zee" type [17, 18, 19, 20, 21, 22] which contributes to r — > fij. Beside it is the similar diagram for Higgs 
production and decay to r/2 at a hadron collider. In the absence of cancellations between the different neutral 
Higgses in the Barr-Zee, it is clear that an upper bound on BR(t — > jj/y) sets a bound on a(gg — > <p — > Tp,). 

There is a large literature on flavour changing observables in the Type III 2HDM. Various textures for the 
Yukawa matrices are discussed in [23] (see also references therein). Recently, the concept of Minimal Flavour 
Violation has been extended to multi-Higgs models [24]. See also [25] for a clear discussion of CP violation in 
a Type III 2HDM that has no tree-level flavour changing neutral couplings. In the Minimal Supersymmetric 
Standard Model (MSSM), Higgs decay to T ± ^i T [26], and its relation to one loop rare r decays [27] have been 
extensively studied (see also citations of [26]). Our analysis differs from [1, 3, 5] in that we have included the 
Barr-Zee diagram in our calculation of r — > fij. This can be relevant if the flavour-changing Higgs has 0(1) 
coupling to the top ("small tan/3"). The current bound on BR(t —> fiy) is also slightly stronger that the 

*E-mail address: s.davidson@ipnl.in2p3.fr 
tE-mail address: g.grenier@ipnl.in2p3.fr 

1 The Higgs sector of the Minimal Supersymmetric Standard Model [9] is a 2HDM. Larger Higgs sectors with multiple doublets 
and singlets[10, 11], and/or triplets, can also be considered. 

2 See, for instance, [15] and references thereto, for a discussion of quark flavour changing interactions in type III models. 
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value used by [5], who performed a more complete study of r decay bounds in 2005. Finally, we are attentive 
to the definition of tan /?; we allow the scalar potential of the Higgs to take its most general renormalisablc 
CP-conserving form, which does not allow a Lagrangian-basis-independent definition of tan j3. The unphysical 
character of tan j3 in type III models has been treated in various ways by previous authors, which makes it 
difficult to compare results. We introduce a "physical" definition of tan/3, via the Yukawa coupling of the r, to 
facilitate comparaison with "almost type II" models such as the MSSM. 




Figure 1: On the left, the "Barr-Zee" diagram which contributes to r —¥ (j,-y. On the right, the production/decay 
diagram for a neutral Higgs <f> at hadron collider. 

In section 2, we introduce our "basis independent" [28, 29, 30, 31] notation for the 2HDM. We discuss 
electroweak precision bounds on the Higgs masses in section 3, and rapidly review bounds (for instance, on the 
Higgs potential) that are secondary in our analysis. Section 4 discusses the contraints from precision flavour 
observables, such as (g — 2) M and r — > /try, and section 5 studies the sensitivity to the 4> — T — fi Yukawa coupling 
of — » rp, at hadron colliders, in the light of the rare decay data. 



2 Notation 

We consider a 2HDM of Type III, in the classification 3 used, for instance, in [28]. Contrary to "type I" and 
"type II" models, the type III scalar potential has no discrete symmetry that distinguishes doublets. When 
Yukawa couplings are included, tree-level flavour changing couplings for the neutral Higgses are possible. In an 
arbitrary choice of basis in doublet Higgs space, the potential can be written [33] 

V = mf^j^i + ml 2 <&\§2 - [m 2 12 $\$ 2 + h.c] 

+ ±A 1 ($ t 1 $ 1 ) 2 + ±A 2 (^$ 2 ) 2 + A3($I$i)($£$ 2 ) + A 4 ($ t 1 $ 2 )($t$ 1 ) 

+ {5A 5 ($I$ 2 ) 2 + [A6($I$l) + A 7 ($5$2)H*2 + ll.C.} , (1) 

where m 2 l7 m| 2 , and Ai, • • • , A4 are real parameters. In general, ra 2 2 , A5, Xe and A7 are complex, but we neglect 
CP violation in this paper for simplicity, and take them <G 5i. A clear discussion of CP violation can be found 
in [34, 30]. A translation dictionary to the form of potential used, for instance, in [8], can be found in [33]. 

The scalar fields will develop non-zero vacuum expectation values (vevs) if the mass matrix m 2 - has at least 
one negative eigenvalue. Then, the scalar field vacuum expectations values are of the form 

<*>-£U). «^C)' 

where v\ and v 2 are real and non-negative, and 

,2^ v 2 +u 2 = 4r^ = (246 GeV)2= 1 tan/3 ^^ (3) 
g \ 2dp v\ 

3 This definition differs from the classification given in [32], where there are 4 types of 2HDM, all of which avoid tree level flavour 
changing neutral currents. 
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The scalar potential of the type III 2HDM, with real couplings, has three parameters in m?-, four in A1..4 
and three in A5..7. One of these parameters can be set to zero by a basis choice (for instance m\ 2 = 0), leaving 
nine independent parameters. The minimisation conditions give v, which is measured, so 8 inputs are required. 
Ideally, one would like to express these parameters in terms of observables, such as masses and 3 or 4 point 
functions. This would avoid confusion stemming from the arbitrary basis choice in Higgs space, and clarifies 
the measure on parameter space to use in numerical scans. 

However, to study the current bounds on "light" Higgses, we only need their masses and couplings to SM 
particles (controlled by (/3 — a) — see the discussion after eqn (5)). So we do not need a complete parametrisation 
in terms of observables; we relate constraints on the scalar potential to the masses in the basis independent 
notation of [28], and otherwise our parameters are mf l ,m H: ,m\, mj 1+ , sin(/3 — a) and the flavour changing 
Yukawa coupling that controls pp — > <fi — > rfi and r — > ^7. 

2.1 Basis choice in Higgs space and tan/3 T 

It is always possible to choose a basis in the Higgs space, such that only one doublet acquires a vev [34]. This 
is known as the "Higgs basis", defined such that (Hi) ^ 0, and (H 2 ) — 0, and in this basis all the potential 
parameters are written in upper case: 

V = M^HlHi + Ml 2 H\H 2 - [m1 2 h\h 2 + h.c] (in Higgs basis) 

+ \K 1 {h\h 1 ) 2 + \K 2 {HlH 2 f + A 3 (HlH 1 )(H 2 i H 2 ) + A 4 (i/ 1 t i/ 2 )(i/ 2 t ^i) 

+ [\K 5 {H{H 2 ) 2 + [AflOffJffi) + K 7 {HlH 2 )]HlH 2 + h.c.} , (4) 

This follows the notation of [28] . 

In the Higgs basis, the angle cxhb rotates to the mass basis of the CP even Higgses h, H: 

h = -(V2ReH^ — v) sinaHB + (V2 KcH 2 ) cosckhb , 

H = (V2RcH^-v)cosa HB + (V^RcH^)sina HB . (5) 

In a 2HDM of type I or II, there is also a choice of basis where the discrete symmetry $j -H- $j, or $j «-> — $j 
is manifest. Usually, the lagrangian is written in the basis where this symmetry is manifest 4 , the angle (3 is 
defined between this "symmetry eigenbasis" and the Higgs basis, and the angle a rotates between the symmetry 
basis and the CP-even mass basis. In which case 

anB = P-a (6) 

and we shall write it as such in this paper. Then the Higgs- W + W~ couplings are igmwC^wwg^" with 

ChWW — S0-a , ChWW = Cp-a , C ' AWW = (7) 

where 

sp^a = sin(/3 — a) , c^_ a = cos(/3 — a) . (8) 
The trilinear couplings between a neutral Higgs and a pair of charged Higgses are —ivC^H+H-, with [33] 

C'hH+H- = ^Sp-a - A-jCp-a , C'hH+H- = ^Cj3-a + , CaH+H- = (9) 

In a type III model, there is no "symmetry basis" , so tan (3 can not be defined from the scalar potential. To 
obtain a type III potential, it is not sufficient to write down a Lagrangian with Xq, A7 7^ 0; one must also check 
that it is not a type II or type I model, in a basis rotated with respect to the symmetry basis. For this there 
are basis independent "invariants", discussed for instance in [28], which vanish in the presence of symmetries. 

In phenomenological calculations, tan (3 usually appears in Yukawa interactions, where it parametrises the 
relative size of the Yukawa couplings and the fermion masses. Whether it can be defined from the scalar potential 
of the Higgses is secondary. Various "definitions" of tan /?, involving fermion masses in the Higgs basis, can 
be envisaged [35, 28, 36, 37]. To maintain the intuition of tan/3 as the relative size of the r Yukawa coupling 

4 For a discussion of Lagrangian basis dependence in the 2HDM, and a formalism that is "basis independent" , see, e.g. [28,29,30]. 
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and y/2m r /v 7 we define H T to be the Higgs that couples to the r, and f3 T as the angle in Higgs doublet space 
between Hi (the vev) and H T : 

H u = Hi sin (3 T + H 2 cos j3 T 

H T = Hi cos f3 T - H 2 sm/3 T (10) 

We choose the r Yukawa, because we are interested in r flavour violation. This reduces to the usual definition 
in a Type II (SUSY) model, where H = ia 2 H* . 

Finally, notice that we pursue a "bottom-up" approach, where we treat tan f) T as a physical parameter, and 
calculate processes which are finite. This allows us to neglect issues related to the renormalisation of tan/3 T 
[35, 38, 39], such as its numerical stability, and additional factors of tan/3 T that may appear. 



2.2 The leptonic Yukawa couplings 



In the Higgs basis for the Higgses, and the mass eigenstate basis for the {ur, cIr, e#, d L , e L }, the Yukawa 
interactions of a type III 2HDM can be written 



V2(qEjHi 



K*,mY 



m 



D 



mi 



um + qhiHi — —d,Ri + — —eRi 

V V J 



+qLiH 2 [p U }ijU Rj + q Li H 2 [p D ]ijd Rj + 1-Li'H.i\p E \i^Rj + h.c. , 



(11) 



where K is the CKM matrix, JH\ = vH^ + ei/°, Hi = ia 2 H* , and the generation indices are written explicitly. 
We are principally interested in the leptons, so we drop the superscript of p E — >• p. 

If the neutral CP-even Higgses are defined to be h and H, with rrih < m#, if A is the CP-odd Higgs, 
and if flavour violating neutral couplings are allowed in the lepton sector only, this gives the following Yukawa 
couplings of leptons in the Higgs mass basis 



s*leptons 
-L, Y 



Ulj 



mi 
v 



-a - -^={[p]ijPR + [p% 3 PL)sp- a 



e- L H 



+±=e i {[p] ij P R - [ P %P L )eA 



+ {u z [{Kp D } t3 P R - [p u ^K] t3 P L ] dH+ +Vi[U^ p] ij P R eH+ + h.c.} 



(12) 



where we included the charged Higgs interactions with the quarks. U is the PMNS matrix, which we henceforth 
drop, assuming that the neutrinos are in the "flavour" basis. For the quarks with flavour diagonal Yukawa 
couplings the Lagrangian can be obtained by taking [p] diagonal, and substituting {p E , e} — > {p D , d} for down 
quarks, and {p E ', e} — > {p u , u} for the ups. 

From eqn (12), The neutral Higgs couplings to fermions are : 



9h,ff> 
9HJf> 
9AJf> 



grnf 
2mw 

V2 



Cp- a Sf,f 



V2 
Pff 
V2 



S(3-a 



(13) 



where gAjf appears in the Feynman rule with a 75. For simplicity, we will take [p E ] hermitian, and normalise 
it 



[p]ij — Ki 



(14) 



It is tempting to expect \kij \ ~ 1 [40]. However, recall the definition of f3 T from the end of section 2.1: in the 
Higgs basis for the scalar doublets, and the mass eigenstate basis of the charged leptons, the r Yukawa coupling 
is 

h [ — Hi + Ptt H 2 ] t r + h.c. = y T l L [cos (3 T Hi - sin (3 T H 2 ] t r + h.c. (15) 
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This gives k TT — tan/3 T so we factor this out and "expect" that Kij ~ 1. Recall that cchb = {0 — a) is a physical 
mixing angle defined from the scalar potential, not the difference of two angles. In particular, the j3 in (j3 — a) 
is unrelated to tan (3 T . 

We are interested in r — p lepton flavour violation, so we allow an arbitrary k t/J/ , and assume that n Te ~ 
~ 0. To perform a general analysis, we should treat the p^tt and p^bb as free parameters, because the angle 

/?/, defined for a fermion / in anology with eqn (15), could be different for each /. However, we attribute Type 

II values: 



\pP]ij = - V2tan/3 7 



TO. 
V 



D 



V2 cot /3 7 



in) 



[p E ] TT = -V2tan^ 7 



v 



(16) 



to all elements of the [p] matrices (except p Tll ), because this parametrisation is adequately representative of the 
cases we are interested in (see the discussion at the end of section 4.4). The expressions of eqn (16) apply in 
the mass eigenstate bases of {cIl, e^, d,R, ur, en}. For a Type III model which has Type II couplings plus small 
corrections (as can arise in Supcrsymmctry[41, 26]), the p— coupling to the top quark is suppressed at large 
tan/3. If in addition, cither sp- a or cp^ a is small, eqn (13) shows that the CP-even Higgs with larger flavour 
-violating coupling, will be weakly coupled to the top. This suppresses both the diagrams in figure 1. 

There are many notations for Type III Yukawa couplings. Our k bears no relation to the one in [28], but is 
proportional to the n of [5](KOT), who define: 



/ / if / L R \ 



KOT 



this paper 



(17) 



The additional power of l/cos/3 in the LFV Lagrangian of KOT causes their LFV rates to scale as tan 6 /3, 
rather than tan 4 j3 T as we find. Such differences must be taken into account in comparing plots. To ensure that 
our results are as "physical" as possible, we plot bounds on (/t T/J tan/3 r ), as a function of Higgs masses, sp- a 
from eqns (6,8) and tan/3 T defined from eqn (15). 



3 Higgs mass bounds 

In this section, we list bounds on the parameters of the scalar potential of eqn (1). These bounds were recently 
presented in basis- independent notation, allowing for CP- violation, in [42], where references to the earlier 
literature can also be found. The bounds can be divided into two classes: firstly, those which apply directly to 
the physical masses and coupling constants. These arise from precision electroweak analyses (the T parameter), 
and are the most stringent. Secondly, there are bounds on the couplings, which follow from imposing that 
WW — > WW is unitary at tree level, and from various considerations about the Higgs potential (positive, 
perturbative...). These must be re-expressed as bounds on the masses. In our "phcnomenological" approach, 
where we allow arbitrary New Physics at the TeV scale, these bounds are less important. We review them 
briefly anyway. 



3.1 Bounds on the potential 

We are interested in bounds on the various Higgs masses. In the Higgs basis, these are related to potential 
parameters as [28] : 
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m 2 H+ = M 2 2 2 + yA 3 (18) 

m 2 A -m 2 H+ = -^(A 5 -A 4 ) (19) 

m 2 H +ml-m 2 A = +u 2 (Ai+A 5 ) (20) 

(m 2 H -m 2 ) 2 = [TOi + (A 5 -A 1 )« 2 ] 2 +4A^ 4 (21) 

sin[2(/3-a)] = -J^jU (22) 

The i? ± mass can be raised high enough to respect b physics bounds (see section 3.2) by increasing M| 2 (the 
decoupling limit [33]). To keep H, A light in this limit requires large As. 
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3.1.1 vacuum stability/bounded from below (lower bound on m^) 

The requirement that the electroweak vacuum be stable, or that the Higgs potential be bounded from below, 
gives a lower bound on nih- See [43, 44] for a review. The constraint is imposed at tree level; one can also 
include radiative corrections and check that potential remains bounded from below. 

Neccessary and sufficient conditions to obtain V(vi — > oo) > 0, in the softly broken 2HDM type II (in the 
basis where Ag = A7 = 0) were given in [33] . A basis- independent analytic discussion of Type II and Type III 
can be found in [45]. However, the Type III bounds do not give simple analytic formulae; a more extensive and 
numerical analysis was performed by [46, 47], who find a lower bound 

m H > 121 GeV . (23) 

3.1.2 triviality/perturbativity (upper bound on m^) 

An upper bound on the Higgs masses (which are cx Ai) can be obtained from requiring that the \ couplings 
remain perturbative at higher energy scales. One can, for instance, impose that the Landau Pole of the Higgs 
couplings be at some sufficiently high scale. The analysis of [46] finds no upper bound on m 2 H AH+ for a softly 
broken Type II, or Type III. 

3.1.3 Unitarity 

In the spontaneously broken electroweak theory, there are delicate cancellations in the tree-level amplitude 
for longitudinal W scattering, between diagrams with Higgs or gauge boson exchange. In addition, at scales 
y/s ^> the Higgs masses, the various scattering amplitudes are proportional to combinations of the Aj couplings. 
S-matrix unitarity gives an upper bound on the S-matrix elements. If the tree level calculation is a good 
approximation to the full S-matrix, then this bound translates to bounds on the Higgs masses. These constraints 
are most interesting for the J = partial wave. This is discussed for the Standard Model in [8, 48], and in 
[49, 50] for the 2HDM (see also the appendix of [42]). 

S-matrix unitarity implies that 1 = SS^= (1 + iT){l — iT^). Writing the S-matrix between states labelled 
by angular momentum J,K 1 .. and all other quantum numbers called j,k..., this gives 

-2Im{(j,J\T\k,K)} = (j,J\TT<f\k,K) (24) 

which, applied to the J = component of an amplitude: 

1 f° 

ao= i^SJ_ s dtA 

implies 2|Im{a }| > |a | 2 . 

There are two limits in which unitarity bounds are usually calculated. The standard calculation assumes 
ntfy <C m| <C s, neglects all masses(0 is an arbitrary Higgs), describes the longitudinal gauge bosons as Higgses 
via the Equivalence Theorem(see [51] for a pedagogical discussion), and obtains bounds on linear combinations 
of the Ai (see [50, 42] for a basis-independent calculation in the 2HDM). Notice that in the strict Type II model 
( m i2 = in the basis where A 6 = A 7 = 0), these bounds are quite sensitive to tan/3 [49]; this may be related to 
the appearance of a massless Higgs, in the limit where one of the vevs Vi — > 0. This is no longer the case when 
m\ 2 ^ is allowed [52, 50]. 

Here, we are more interested in the limit rrvfa <C s <C m|, where <j> is some of the Higgses. That is, we allow 
arbitrary New Physics at the TeV scale to preserve S-matrix unitarity in the rriyy <C <C s limit, but we 
would like to know the bounds arising on mass splitting among the Higgses, at yfs < TeV, when some Higgs 
are heavier than y/s, and some are lighter. 

The gauge contribution to longitudinal W scattering (no Higgses exchanged in s or t channel), is [48] 

■ 2 

A{W L W L ^W L W L ) = ^-{s + t) + ... no Higgs. (25) 

Requiring ao < 1 implies the scale of Higgs masses should be < TeV. In the case where some Higgses are light, 
for instance rrih < s, including them in the amplitude reduces the coefficient of s + t, and raises the upper bound 
on the masses of the remaining Higgs. We conclude that unitarity constraints do not give us relevant bounds 
on the mass differences among the Higgses. 
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3.1.4 Electroweak precision tests 



New physics that couples weakly to Standard Model fermions, but has electroweak gauge interactions, can be 
constrained by the measured values of the "oblique parameters" [53, 54]. If the vacuum polarisation tensor 
between gauge bosons i and j is defined as 

TVi»{ q )=g^A ij tf) + q » q »B ii tf) (26) 

then the parameters S and T can be defined as [11] 

Azz(m%) - A zz (0) 8A J7 



S 4s^c^ m| ^ dq 2 



c w s w 3A~/Z 



q 2 =0 s w c w dq 2 



(27) 



q 2 =0 



f = aT = Aww 2 {Q) - ^# (28) 

where % = sin6w The Standard Model contributions to these parameters, including that of the SM Higgs, are 
assumed to be subtracted out. The contributions to S, T, and U, (as well as V, W and X), due to an arbitrary 
number of Higgs doublets and singlets have recently been calculated in [11]. S, T and U were calculated in the 
CP violating 2HDM in [42]. We use here the formulae of [11], which verify the previous calculations of [55, 56] 
for the 2HDM. 

As discussed in [42], the 2HDM contributions to S and U tend to be small enough, but the contribution 
to T can exceed the value allowed for New Physics ( —0.15 < T < 0.20 at one er [53]). We impose the bound 
—0.05 < T — S < 0.10 at one cr [53]. The calculation of T in multi-Higgs models is presented in detail in the 
first paper of [11]. They give 

T = 1 2 {F(m 2 A ,m 2 H+ ) + c 2 _jF(m 2 H+ ,m 2 )-F(m 2 A ,m 2 )} 

+s 2 1 _ a [F(m 2 H+ ,m 2 H ) - F(m A ,m 2 H )} 

-3c}_ a [F(m 2 z ,m 2 ) - F(m 2 w ,m 2 h ) + F(m 2 w , m%) - F(m 2 z , m%)] } (29) 

cv ^ x + y x y i x fo. n \ 

F {x,y) = — In- 30 

2 x-y y 

is a positive function, that vanishes for degenerate masses. So as masses in the loop split, the contribution 
increases. 

It is well known, and clear by inspection, that T becomes small in various limits, such as m A — > niH+, or 
ran — > rn,H+ when sp- a — > 1. When studying h and H production and decay at colliders, we will assure the 
precision constraint by imposing tua — itlh+ ± 10 GeV. For h production, we do not consider the parameters 
ran — > niH+ and S/3_ a — > 1, because a(gg — > h — > r*^^) cx s 2 _ a c 2 _ a . The collider cross-section for the 
pseudo-scalar A is not suppressed by Sf)~ a or c^- a , so when we study cr(gg — > A — > r^/i^), we can ensure the 
precision constraint by requiring mn — rn,H+ ± 10 GeV and s^_ Q — > 1. For m A > 100 GeV, T - S is within 
the 1 a allowed range for 7r/3 < (3 — a < n/2. 



where 



3.2 Flavour physics bounds 

The charged Higgs H + neccessarily has tree level flavour-changing couplings, like the W + . Its mass is therefore 
constrained by various flavour-changing observables, such as b — > sj and B + — > t + v. The various bounds have 
been discussed in [57], who find m H+ > 250GeV for 2 < tan/3 < 20, and m H+ > 300Get^ for tan/3 < 50. 
This limit is lower than that of Misiak et al. from b — > s-f [58] , which is mn+ > 300 GeV (at 2a) , because of 
differences in the procedure of extraction of the bound. 

The charged Higgs contributes at tree level to the decay of pseudoscalar mesons M. Since the SM W- 
mediated amplitude is helicity suppressed, the additional suppression of the Higgs amplitude is only a factor 
~ (wim /niH+) 2 [59}. Charged meson decays, such as B + — > t + v, therefore constrain the mass and couplings 
of H + in type I and type II models. In addition, they can constrain the flavour-violating couplings of type III 
models. It was shown in [60], that a precise determination of Rk = T(K~ — > ev)/Y{K~ — > fj,P), by the NA62 
experiment, could be a sensitive test of the p T ^ coupling. 



7 



3.3 Summary 



We retain two constraints from this section: mn+ <; 300 GeV from B physics (as discussed in section 3.2), and 
the T parameter will be small enough in two cases: either tua — mn+( we take numerically itia = rriH+ ± 10 
GeV, or ran — mH+, with j3 — a > 7r/3 to ensure cp^ a — > 0. 



4 Constraints on flavour changing Yukawa couplings to leptons 

In this section, we include the lepton flavour-changing Yukawa couplings of the Higgses, see eqn (12). We 
consider mass ranges for the neutral Higgs which are consistant with the constraints discussed in the previous 
sections, and study the sensitivity of (g — 2) M , r — >• p.'y and r — > r\p to the the flavour-changing coupling p Tfl , 
introduced in eqn (14). 

A systematic study of bounds on a 2HDM (Type II), with lepton flavour violating Yukawa couplings, has 
been performed in [5]. They impose constraints on p Tfi arising from tree and one-loop contributions to rare 
decays, then study the 4> —> t\x decay rate at colliders. As this is a Type II analysis, tan/3 appears in the 
formulae. In the context of supcrsymmetric models [27], it was shown in [68] that the bounds from one and 
two loop contributions to r — »■ £7 are more stringent than those from r — )■ 3£ (except if the Higgses are very 
degenerate). 

The anomalous magnetic moment of the muon can also be sensitive to a 2HDM with flavour- violating Yukawa 
couplings [3, 4, 69, 71]. For p TfJ _ ~ lO^/rrvm^J/w, the 2HDM can fit the current discrepancy in (g — 2)^. However, 
as we will show, the constraint from t — > ^7 precludes the explanation of the (g — 2) M discrepancy, for large 
areas of parameter space. 

4.1 The decay r — > -qji 

It was shown in [5] , that t — > r\p gives a relevant constraint on n Tfl for light pseudoscalars A. We briefly review 
here their discussion, using the updated bound [61] BR(t — > -qp) < 6.5 x 10~ 8 . 
The branching ratio is 



BR(t pvv) \mAj m 2 . TM (m u + m d + 4m s ) 

where BR(t — > pvv) = .17, and f v ~ from [62]. The current experimental upper bound gives 

|K - |tan2 ^ <7 HiocT^7) 2 ^ 

where we approximate the final fraction of (31) as tan 2 /3 T /4. The bound (32) agrees with the result given in [5]. 
This is comparable to the bound from r — ► pj (see figure 3); we see that a light A is allowed for small tan /3 T . 

4.2 The dipole effective operator 

Bounds on a large class of effective operators that change r — p flavour are presented in [62] . Several bounds 
arise from the dipole operators, which can be included in an effective Lagrangian as [63] 

-5— {H)eia af} P Rej F aP + h.c. (33) 



A 



where i, j are the flavours of the external leptons. The chirality flipping operator must contain an odd number 
of Yukawa couplings, including p T ^ and a flavour diagonal Yukawa coupling of the external Higgs vev. The 
model-dependent coefficient /A? NP , can be related to a New Physics contribution Sa^ to the anomalous 
magnetic moment of the muon, and to the Al^r factors that appear in r -> /17 [63]: 

eda^ _ Re{C^} v em T A T £ _ C T ^ v em T A T £ _ C T * v 

4^ ~ A 2 NP 71 2 ^A^Tl 2 ~ A 2 NP 71 

The Al,r appear in the r — > /17 branching ratio as: 

r(r -> M7) _ 48A 2 a 4.4 x 10^ 
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where we used the current r — > ^7 bound [64] (very similar to the BR < 4.5 x 10 8 of [65]) to obtain 
\A L \ = \A R \ < 10- 4 G F . 

The current experimental and theoretical determinations of (g — 2) M are[53]: 

(g - 2) M _ _ f 11659208. 0(5.4)(3. 3) x lO" 10 expi 

2 j 11658471.810(0.016) + 691.6(4.6)(3. 5) + 15.4(.2)(.l) x 10- 10 QED + hadronic + EW 

(36) 

where the first experimental uncertainty is statistical and the second systematic. The hadronic uncertainties 
are from lowest and higher order. The electroweak contribution, which at one loop is 



n EW,l-loop ^ GFm 2 ^ 
~ 8V2V 



§+!<i-^> 



(37) 



includes the two loop effects (a <~ 25% correction), and the electroweak uncertainties are from the unknown 
Higgs mass and quark loop effects. The difference between experiment and theory is 

Aa^ = a e * p - al M = 29.2(6.3)(5.8) x lO" 10 (38) 

so when we ask our model to "fit the (g— 2)^ discrepancy" , we will ask it to contribute Sa^ ~ w ~ +15 x 10~ 10 
to the theoretical calculation of a M . 

We now calculate the bounds from a selection of one and two loop diagrams contributing to (g — 2)^ and 
to t — > /17. We aim to describe the leading constraints. The model we consider has many unknown Yukawa 
interactions, so which diagrams give the most "interesting" limits depends on what is assumed about the pattern 
of Yukawa couplings. For instance, a two-loop diagram that is linear in a small Yukawa coupling can give a 
more significant bound than a one loop diagram that is quadratic in that same Yukawa coupling [17]. Various 
theoretically motivated patterns for the Yukawas have been discussed in the literature [40, 41, 66], and many 
analyses concentrate on the supersymmetric case, where the flavour-violating Yukawas are loop induced, and 
p tt oc cot/3, p Tfl oc tan/3. 

We only include diagrams that contain the flavour changing couplings 5 p Tjl . It appears squared in the 
lepton flavour conserving amplitude a^, and in the lepton flavour changing rate T(t — > ^7). 

At one loop, three Yukawa couplings are required on the fermion line of the dipole operator, so we expect 
oc \g(j, Tf i\ 2 m T and C^ T oc g^rg^T^r- We do not consider possible large logs that could arise from 
electroweak corrections to these diagrams. We do consider two loop contributions to C TAi which have only one 
Yukawa coupling on the lepton line. The other end of the <f> propagator must attach somewhere, and an external 
Higgs vev is required for the dipole operator. If this pair of Higgs couplings can be large (for instance gauge, 
third generations Yukawas, or scalar self-interactions) they could compensate the l/(167r 2 ) relative to the one- 
loop diagram. Therefore we include the two-loop "Barr-Zee" diagrams (figure 1) with third generation quarks, 
and with W bosons. We neglect "Barr-Zee" diagrams with charged Higgses in the second loop for simplicity, 
and because these loops are relatively suppressed 6 . 

Our formulae are incomplete, possibly gauge dependent (the W loop diagrams of figure 2), and we have not 
excluded large logs in higher loop corrections (as in the SM electroweak contribution to g — 2 [67]). Due to 
these uncertainties, we will be reluctant to exploit cancellations between diagrams. 



4.3 g-2 

There are one loop (and higher order) contributions to mediated by the flavour conserving couplings of the 
2HDM. The possibility of explaining the (g — 2) M discrepancy with these terms was discussed in [19], who show 
that there is little parameter space consistent with other constraints on the 2HDM. We assume that (g — 2) M 
does not constrain the flavour-conserving parameters of our model[57]. 

In the presence of the flavour changing p TfJi coupling, there is a one-loop contribution to (g — 2) M , illustrated 
on the left in figure 2. As discussed above, it should be of order the one- loop electroweak gauge contribution, or 

5 So we neglect the flavour diagonal contributions to (g — 2) — see the references at the beginning of section 4.3. 

6 Such diagrams could be expected to give significant contributions, because they are proportional to quartic Higgs couplings, 
which can be ~ 1 because we have mass 2 differences in our Higgs spectrum ~ v 2 . However, the amplitudes are suppressed by 
m 2 /m| f+ , and there is a partial cancellation between the loops with both photons attached in the same place and with the 
two photons attached separately [22]. The loops are therefore an order of magnitude smaller than the top loop, for quartic 
Higgs couplings ~ 1. Secondly, the relevant Higgs couplings (see eqn (9)) A3 and A7, are independent of the Higgs masses (see 
eqns (18) to (22)), so arc free parameters which just add more confusion to the analysis. We are reluctant to allow | A.3 1 , IA7I ^ 3, 
which would allow to (partially) cancel the top loop. 
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Figure 2: One and two loop diagrams, which contribute to the dipole operator coefficient Cll, (see eqn (33)). A mass 
insertion is required on the fermion line of the one loop diagram. For k = r and j = i = /u, these diagrams depend only 
on the Yukawa couplings p Tll . 



less. The chirality flip is provided by the t mass, which increases this diagram with respect to flavour diagonal 
ones. Neglecting the lepton masses in the kinematics, the one loop contribution gives [3] 



2 

a 2HDM,LFV ^ „2 m ^ m T / Am ' 



87T 2 J 




(39) 

■ O'/l "lll^ \ III' L I 

where <f) = h,H, A, and is from eqn (13). 

In the absence of cancellation between the opposite sign CP-even and CP-odd Higgs diagrams, 

^^<32^ (40) 

contributes at most the experiment-theory discrepancy in Aa M ~ 2a^ w [3] . This bound agrees with the naive 
power counting expectation that ^\Pt^\ 2 /m 2 ^ <Gf- 

4.4 t — y yWy 

The two body decay r — > /try can arise via loops, and the current bound on the Branching Ratio is given in 
eqn (35). Since the Standard Model decay t — > Ivv is three body, the bound on New Physics in a loop is 
particularily restrictive, because the |l/167r 2 | 2 loop factor is partially compensated by the two to three body 
phase space ratio. 

To estimate BR(r — > fi-y) in our model, we assume that Al = Ar = A, and neglect the lepton masses 
in the kinematics. We include the one- loop diagram of figure 2 (the first sum below), and a subset of two- 
loop diagrams. Following [22], we include the two-loop diagram of figure 1, with an internal photon and a third 
generation quark. This is the second sum below, with f = t,b, and is gauge invariant on its own. The remainder 
of eqn (41) is a subset of the two loop W diagrams (as in figure 2) [22]. The amplitude is: 

A ~ 




^ N c Q 2 f a l m 2 f 



4h,H 2nm T m w 



„2 no ™2 



S A<^> + T^> + ?<^> + "* 2^" 



(41) 



where <j> — h, H, A, f = t,b, the coupling g^ff of the internal loop fermion to the scalar <p is given in eqn 
(13), and the scalar- VF + TF~ couplings C^ww are given in eqn (7). There is a factor m T in the denominator 
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of the two-loop expressions because it appears in the definition (34) , and a factor of the loop mass because the 
functions f(z),g(z),h(z) are proportional to this mass 2 , whereas the loop has a single mass insertion. 
The various functions are [22] : 

z f 1 1 x(l — x) 

fA(z) = g(z) = - / dx—- r In pseudoscalar (42) 

2 J x(l — x) — z z 

t t ^ z f\ (l-2x(l-x)) x(l-x) . 

fh,H{z) = - dx^— ^ ^ln^ '- scalars, (43) 

A Jq XyL ' X J Z Z 



and 

z f 1 dx 

h{ Z ) : 



2 J x(l — x) — z 



1 , \ \U X{1 - X) 



x(l — x) — z 



(44) 



They are <~ z for arguments z of order 1, and for small z, /^(z) ~ §(lnz). 

The t — > ^7 amplitude A depends on tan/3 r , (/3 — a), and the Higgs masses. It is useful to estimate the 
relative size of the one-loop, two-loop fermion, and two-loop W contributions in various cases. 

1. We start by estimating the one- loop diagrams contributing to r — > /try. In the absence of cancellations 
between the opposite sign CP-even and CP-odd Higgs diagrams, the approximate r — )■ /L17 bound |A| < 
10~ 4 G F gives 



r K rT tan 2 /3 T /m M m 2 



m 



2 

— ^jhi^<10- 4 (45) 



87T 2 Y m T m rf, m T 

or, for k tt ~ 1: 



k mt tan 2 p r < 160 . (46) 

V m 4> J 

which is weaker than the estimated (g — 2) bound of eqn (40) for small tan/3 T , and more restrictive as 
tan/3 T grows. In the case <j> = A, the bound (32) from r — > 77^ is more restrictive. 

Consider now the two- loop contributions to t — > ^7, starting with the top loop. For small tan /3 r , or large 
mixing s^j_ a <~ cp- a between the Higgses, this can be the dominant contribution to r — > /try. The ratio of 
the top-loop to one loop amplitudes, induced by a particular Higgs <j>, is 



2loop top aQl{cp- a sp-a + cot /3 T )m 



Hoop m T vm^ m T n TT tan/3 T ln(^p) 

n 2 (cp- a s P - a +cot/3 T )m 2 1 

~ ^ rnl KTT tan/3 T ln(^) ^ 

where 50//' is from eqn (13). So for n TT ~ 1, and cp_ a sp_ a <~ 1, the 2-loop top diagram dominates over 
the one loop for tan/3 T <^ mt/rrib. This is because all the Higgses have an 0(mt/v) coupling to the top, 
independently of tan/3 T . However, the c^aS^a terms of the h and H amplitudes have opposite sign 
(recall that gAff is independent of /3 — a). Imposing the approximate r — > ^7 bound |A| < 10~ 4 Gf on 
the top amplitude, for case 1: mu — tuh -C iha, and for case 2: nih — rriA <C mjj with sp- a ~ 1, implies 

10 >{ ( !r ^g^ sin2 (/ ? -") + t5^r) ^tan/3 r easel (4g) 
I i^t case2 

which is somewhat more restrictive that the estimated (g — 2) bound of eqn (40), and one- loop r — > /X7 
bound of eqn (45). However, for large tan/3 T , small c^aS^a < cot/3 T and k tt <~ 1, the one-loop 
contribution is larger.. 

Superficial inspection suggests that the two loop diagrams with an internal b or W loop, never provide a 
dominant contribution (although they are included in our plots). Parametrically, the two loop contribu- 
tions of the t, W and b give the following three terms 



k t/j tan j3- 



- ( N c Qtmt[cp- a sp- a + cot P T ] + cp-asp-amw + N c Qlml tan /3 r ln(^) 

V m b , 



(49) 
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so one sees that the W loop can be neglected with respect to the tops. Notice from eqns (13) and (7), that 
when sp^ a or c^_ Q is small, one of h or H is "SM-like", meaning that it has coupling ~ gmw to W + W~ , 
flavour-diagonal couplings ~ m,f/v to the fermions, and suppressed flavour-changing interactions. So the 
effective interaction of W + W~rp,, induced by the neutral Higgses, is oc sin2(/3 — a). Similarly, the Higgs- 
induced tirp, is oc sin2(/3 — a) + O(p^tt)- This is in accordance with [22], who observe that the W loop 
contribution vanishes in the decoupling limit due to unitarity arguments. It disagrees with the MSSM 
analysis of [68], where the W contribution is included but the tops are not. 

Comparing the one loop amplitude to the two loop bottom amplitude, for large tan/3 r < l/(cg_ a S£_ a ) 
when the b loop could dominate the top, one obtains: 

Hoop k tt tan /3 T m T ln(^) m T vm 2 ^ n TT m 2 T 

21 oop bottom vm 2 ^ aQlml tan/3 T m(^-) aQlrn 2 

This suggests that unless n TT -C 1, the one- loop contribution is larger than the two- loop when l/(c^_ a s^_ Q ) > 
tan/3 r > 3. 

4. Finally, recall that cancellations can occur in the sum. There is a relative negative sign between the 
CP-even and CP-odd Higgs diagrams (which has little effect in many of our plots because we set mj ~ 
mu+ — 300 GeV, to minimise the T parameter), and also between the h and H induced two loop top 
amplitudes at negligeable cot (3 T , which would cancel for m\ — m 2 H . 

The bound on k t ^ tan,5 r from r — > ^7 is plotted in figure 3, for mt = 117 GeV, mH = 130 GeV, mn+ = 300 
GeV, \mA — mu+\ = 10 GeV, and k tt = 1. As expected, for tan/3 r > few and small sin(/3 — a), the amplitude 
is dominated by the one-loop contribution, and the constraint on K T/J tan/3 T scales as l/tan/3 r for fixed /3 — a. 



bound on tan(beta) kappa ( 




Figure 3: Bound on K T/ j, tan /3 T from the experimental limit on BR(r /^'y), as a function of tan j3 T and sin(/3 — ct), for 
m h = 117 GeV, mH = 130 GeV, m H + = 300 GeV and \m A - m H +\ < 20GeV. We take n TT = 1. 

The relative importance of the (g — 2) and r — > fij constraints is illustrated in figure 4, for a particular 
choice of Higgs masses. The two loop contributions to r —> ^7 (discussed above) are also included in this plot. 
For K Tf2 = 1, the double ratio of the predicted a 2 H DM < LFV from eqn (39) over the (g — 2) discrepancy (taken to 
be 15 x 10~ 10 ), is divided by the predicted r — > ^7 branching ratio over the current bound: 

n 2HDM,LFV 



BB 2HDM.LFV^ T ^^ (51) 
BR data (T^fJ,-f) 

Since both predictions scale as |p T ^| 2 , this cancels in the ratio, which therefore quantifies the relative sig- 
nificance of the bounds. Since the ratio < 1, we conclude that for generic mass choices that give a small T 
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parameter, it is not possible to take p Tfi large enough to fit (g — 2)^,, without exceeding the bound from r — > prf. 
The plot is obtained by a grid scan. We consider f3 — a : ir/4, because the range ir/4 — > ir/2 is equivalent 
if one simultaneously exchanges and m#. 



g-2/tmg | 




Figure 4: The double ratio, given in eqn (51), of the contribution to (g— 2) M divided by half the experimental discrepancy, 
over the contribution to r — > /17 divided by its experimental limit. It is plotted as a function of tan f3 T and sin(/3 — a), for 
rrih — 117 GeV, mn = 130 GeV, rriH+ = 300 GeV, and \mu+ — tha\ = 10 GeV. Both contributions are oc («v M tan/3 r ) 2 , 
which cancels in the ratio. Assuming that our mass choices are representative, this shows that the current experimental 
bound on r (J7 is more restrictive than (g — 2) M . 



4.5 Assumptions about the pff 

As previously mentioned, the bounds on p r{i depends on the assumptions made for the other model parameters, 
such as p TT , ptt, Pbb, and the Higgs masses. We focus on three limits. 

The most conservative approach to setting bounds on p T[1 , is to neglect all other couplings. In this case, the 
(g — 2) bound will hold, and vary as a function of the neutral Higgs masses, but the r — > /17 bound plotted 
in figure 3 does not apply because it assumed p matrices as given in eqn (16). Neglecting the flavour diagonal 
Yukawa couplings of the flavour violating 0, is equivalent to setting them to zero. So this case corresponds to 
cp- a = 0, where one CP-even Higgs (h) is SM-like, and both H and A have the flavour-violating p Tfl coupling, 
but no tree-level interactions with ts, bs or Ws. Such additional Higgses would be difficult to produce at colliders, 
so we do not consider further this case. 

We are interested in a Higgs that could be copiously produced at hadron colliders in gg fusion. This can 
be obtained by allowing the flavour-violating Higgs an 0(1) Yukawa coupling to the top. This is realised for 
sin2(/3 — a) ~ 1, in the parametrisation of Yukawa couplings given in eqns (13) and (16). In this case, the top 
loop contribution to r — > /17 (see figure 1) is significant at small tan/3 r , as discussed in section 4.4 and the 
diagram for the gg — ¥ rp process at colliders is closely related to the r —¥ pj diagrams, as expected from the 
diagrams in figure 1. 

Finally, consider the large tan /3 T limit. Recall that we defined tan /3 T in our type III model using the tau 
Yukawa coupling (see eqn (15)). Then we assumed that all the other fermions shared this definition of tan/3 T , 
and had type II Yukawa interactions. Such "almost Type II" couplings could arise in Supersymmetry, where 
the flavour violating Yukawas grow in the large tan /3 limit [26] . For large tan /3 T , the bound on K Tf _i tan (3 T arises 
from the one loop contributions to r -> ^7, and is (approximately) independent of sin2(/3 — a). However, the 
Higgs-induced ttrp interaction is oc sin2(/3 — a), so, as we will see in the next section, gg — > <f> — > ^ p T is 
suppressed at small sin2(/3 — a). This can be seen by comparing figures 5 and 3 at small 8 sp- a . 

7 The Higgs couplings are unchanged if one interchanges h with H, and simultaneously (/3 — a) — ¥ (f3 — a) — it/2. And since h 
and H are summed in the amplitude A: A(cp_ a — ¥ 0) = A(sp_ a — > 0)+ corrections due to m? H ^ m^. 

8 Small sin2(/3 — a) can be obtained because either sp_ a or cp_ a is small. Both cases are discussed in the supersymmetric 
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4.6 Summary 



For much of the parameter space of this model, the bound from r — > /17 precludes explaining the g— 2 discrepancy 
(see figure 4). We will see in the following section, that generic 2HDM parameters giving a detectable rate for 
pp — > <f> — > t fj^ in current Tevatron data are already excluded by precision bounds. However, a signal at the 
Tevatron could arise if there are cancellations in the r — > ^7 amplitude. 



5 Colliders 

The leading order production cross-section of Higgses, by gluon-gluon fusion at a pp collider, is [9] 



vlo(pp-^ <t>) 



288\/27rs 



q=t,b 



1 dx 



g{x)g{x/z (j ,) 



where z$ — rri^/s, z q — (4m 2 )/m 2 , the gluon density in the proton (or anti-proton) is g{x), 

A h ^ H {z) = 2z[l + (1 - *)/(*)] 
A A (z) = 2zf(z) 



and 



arcsm 
log 



1-2 



z > 1 
z < 1 



(52) 



(53) 
(54) 



(55) 



The next order QCD corrections to the production cross section are ~ 20 — 90%, and can be mimicked by a K 
factor [9]. Finally, to obtain the cross-section for pp — ► Tp, the cross-section (52) should be multiplied by the 
branching ratio BR(cf) — > Tp). 

The Tevatron searches for SM Higgses, decaying to rf , in the mass range 105- 145 GeV [72, 73]. A recent 
DO analysis [73] obtains limits on the cross-section x branching ratio of order 20-90 x the SM expectation. If 
we imagine that the Tp final state is detectable with similar efficiencies to rf , then the parameters to which the 
Tevatron is sensitive can be estimated as follows. We normalise the <f> production cross-section and branching 
ratio to the SM expectation for hsu — > t+t~: 



_ cr LO {pp^(f>) 



2 V 

4 Z^/q=t,b 



Z q ) 



R. 



BR 



olo(pP -> h SM ) 
BR(<p — > Tp, /if) 



(56) 



BR(h SM -> tt) 

2k 2 tan 2 /3 T m M /m T 



(1 - B 2 w) + \c<t,ww\ 2 B2W 



b P-a 
r 2 

C 0-a 



> = h 
= H 
• = A 



(57) 



where hsu is the Standard Model higgs, rrihSM is taken to be equal to and B 2 w = BR(hsM — > W + W ) 
which varies with m/j. We estimate that 

JZJiZf fl < 30 (58) 

because the Tevatron limit on cr(pp — > /ism — > ^f) is ~ 20 — 90 x the SM expectation 9 . In the large vha 
scenario, where the Tevatron would be looking for a CP-even (f>, our lepton flavour violating rate is maximised 
for large sin2(/3 — a) <~ 1 and small tan/3 T = 2. The Tevatron bound would be of order 



K TM tan/3 T £ ./ — (20 to 90) - 20 



(59) 



which is on the border of the exclusion estimate from r — > /17, given in cqns(45) and (48). In the large to# 
scenario, where s^_ Q — > 1, the Tevatron could look also for A — > r^fj,^. At small tan/? r , the production rate is 
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sigma/tmg^ 




tan beta 



Figure 5: Estimate of the relative sensitivity of a(pp — > h — > r ^) and r — \ fjrf to p TM from current data. The plot 
is the double ratio of R^R^ R (see eqns 57 and 56) over BR(t — > /17), divided by current experimental limits (see eqns 
(35) and (58)) , as a function of tan/3 T and sin(/3 — a) (« rM tan /3 T cancels in the ratio). The plot is for h production at 
colliders, with mh = 115 GeV, van = 130 GeV, mu+ = 300 GeV and \rriA — mn+ = lOGeV. Since the ratio < 1, r — > fi'f 
is more sensitive. Alternatively, this plot indicates the amount of tuning required in the r — > ^7 rate to accomodate a 
signal for h — s> Tfi at the TeVatron in the near future. 

similar to the standard model, so the bound should be of order eqn (59); for large tan/3 T , A production in gg 
fusion is suppressed. 

A more credible estimate of the relative sensitivity of colliders and t — > /17 to the parameter K TAl tan/3 T , 
could be obtained from a double ratio, as discussed for (g — 2) and t — > /17 at the end of section 4.4. We plot in 
figure 5 the double ratio of R^R^ /[the current bound on hsM — > rf], divided by the predicted BR(t — > ^7) 
normalised to its current experimental bound. Both rates are oc (k tm tan/3 T ) 2 , so this cancels out of the ratio. 
This plot is made for the production, and decay to r ± fi T , of a CP even Higgs h of mass 115 GeV, with 
niH — 130 GeV, rriH+ = 300 GeV and \m,A — mn+\ = 10 GeV. If tua — itih+ <: 250 GeV for B physics and 
the T parameter, then the current bound from r —> /i7 makes it difficult to detect h — > rjl at the TeVatron 10 . 
The sensitivity to H — » rfi is worse, in the interesting sin(/3 — a) ~ cos(/3 — a) ~ tan/3 r region, due to sums and 
differences among these angles all of the same orders. However, a hadron collider able to detect hsM — > tt (an 
improvement of ~ 30 with respect to the current Tevatron limit), would be more sensitive than r — > fj/y in the 
sin2(/3 — a) > 1/2 and small tan/3 r region. Also, cancellations are possible in the r — > /17 amplitude, which 
would increase the sensitivity of colliders relative to rare decays. 

There are dubious approximations in obtaining this plot. We neglect the contribution of the (f> — > t\x 
decay in computing the total 4> decay rate (this approximation was also used to obtain eqn (59)). We make 
this overestimate of the branching ratio, so as to obtain a formula which is oc |<?0i-/i| 2 , so easy to compare to 
BR(t — > ^7). It is reasonable where r — > fij imposes K TAt tan/3 T < 3 (because then T(cj) — > r/2) < T(h — > rf), 
and the contribution to the total decay rate is not to important); however, in the interesting small tan/3 r and 
large sin2(/3 — a) region, it could overestimate <r(pp — >• h — >• r ft* by a factor ~ 2 (see figure 3). A second 
doubtful approximation is that the higher order QCD corrections cancel in eqn (57). This may be acceptable 
when cf> and hsM are emitted from a top loop, but in the large tan/3 T limit, the b loop can also be important for 
4> production, and its NLO corrections are different. Finally, the experimental bound we use from the search for 
hsM — > tt [73], allows for several Higgs production mechanisms: associated production with a W or Z, vector 
boson fusion, or gluon fusion, and assumes a final state of rf+ 2 jets. Whereas in (57), we assume production 
by gluon fusion. 

analysis of [68]. Recall that c^„ a — > in the decoupling limit [33], where A, H and are "heavy" (J> 2m-w), and the light higgs 
h has almost SM couplings. 

9 we use figures from table VI of [73], obtained with ~ 5fb~ 1 of data 

10 To rescale this plot for the LHC is straightforward: multiply by the TeVatron limit of ~ 30x SM expectation for h$M t + t~ , 
and divide by the LHC limit on cr(gg — > tj> — > ^) / u{gg — > h$M — > r+r~). 
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We can also compare our prediction to bounds [74] on a MSSM Higgs produced via gluon fusion, and decaying 
to t+t~. This analysis assumed one r decaying hadronically and the other one to a muon and neutrinos. The 
cross section limits on a x BR(<p — > tt) displayed in figure 5 of [74] indicate the current TeVatron limit on 
gg 4(^4 t\i. In <f) — > r ± /x =F , the muon is directly produced in the decay of the Higgs, rather than from a 
r, so the signal would have a global detection efficiency roughly 5 times higher than the signal of [74], due to 
the absence of the factor 0.17 coming from the decay rate of r in fi. So the y-axis in figure 5 of [74] could 
roughly be labelled as a x BR(<fr — > t/i)/0.17. For a Higgs of 115 GeV, this gives a limit around 30 times the 
SM expectation, which is similar to the limit quoted in [73] with twice the luminosity. 

For experimental bounds, we have extrapolated the potential reach of h — s- t\i searches from the published 
searches for h — > tt. In [3], it has been shown that by using appropriate cuts, it was possible to distinguish 
between h — > r/x and h — > tt signals (see figures 13 and 14 in [3]). Various cuts can help extract a h — > t/i 
signal from the standard model backgrounds that are still selected by a h — > tt analysis. Increasing the muon 
Pt threshold is a first option. In [3], this threshold is at 20 GeV, already above the thresholds used in h — > tt 
searches (10 GeV for CdF and 15 GeV fo DO [75]). With Higgs mass above 100 GeV, increasing the threshold up 
to 30 GeV can safely be done. Figure 1 of [74] shows that threshold increase would greatly reduce the amount 
of standard model backgrounds remaining in a h — > tt analysis. The dominant remaining backgrounds would 
be Z-» tt, and W+jets. 

Another potentially efficient variable to isolate h — > r/x signal is the effective transverse mass of the tau mu 
system as defined in equation (30) of [3]. For W and Z background, this variable would peak at the W/Z mass 
while for higgs signal, it would peak at the higgs mass (see figure 11 of [3]). Hence, a cut on this variable could 
further reduce the background especially for higher higgs masses. 

Compared to the h — > tt analysis, a dedicated h — > Tfj, analysis could then have fewer background events 
for a similar selection efficiency leading to sensitivity and limits which would be better than the one crudely 
extrapolated from current experimental limits on h — >• tt searches [73, 74]. 



6 Summary 

At hadron colliders, an interesting signature of New Physics would be the production of a neutral Higgs <f>, 
followed by its decay to T^pi^ . For <C 2mw, the branching ratio can be large, and the process could arise 
in a wide variety of models. Such New Physics is constrained by precision observables, and the upper bounds 
on rare decays such as t — >• ^7. In this paper, we are interested in the implications for collider searches of these 
loop contributions. 

We parametrise the New Physics at mass scales < 400 GeV, as a (CP- conserving) 2 Higgs Doublet Model 
(2HDM) of Type III, meaning that we allow our neutral Higgses to have the tree-level lepton flavour changing 
couplings of eqn (12): 

lmu,m T 

~ K TM tan/3 T W —2 — {cp- a h - sp- a H + lA) t\x 

Our choices of notation, basis in Higgs space, tan/3 T , and parametrisation are discussed in section 2. We will 
quote and plot bounds on K T/J tan/3 T , to avoid artificially strong bounds on k t/j at large tan/3 T . We study the 
phenomenological constraints on this model: we admit arbitrary New Physics at scales <~ TeV, and retain the 
contraints on the 2HDM arising from the T parameter, b — » 37, (g — 2) M and r —¥ ^7. 

The T parameter and b — > sj restrict the flavour-independent parameters (masses) of the model. The 
charged Higgs of the 2HDM gives a significant same-sign contribution to the Standard Model (SM) amplitude 
for b — > S7 (see sect 3.2). For m# + > 250 — 300 GeV, this contribution is within current experimental error. In 
this paper, we assume such a heavy H + ; this has little effect on the neutral Higgses we are interested in. The 
T parameter, discussed in section 3.1.4, is the sum of many terms of different sign. One way to ensure that it 
is small enough is to take tua ~ tuh+, as we assume for most of the plots. However, other cancellations are 
possible within the 2HDM (such as run ~ with sin(/3 — a) — >• 1), or additional new light particles (such 

as arise in supersymmetric models) could contribute to T. 

The decay t4/j7 and the anomalous magnetic moment of the muon (g — 2)^ constrain the flavour-changing 
coupling K TA1 tan/3 T , and are discussed in sections 4.3 and 4.4. The neutral Higgses h, H and A contribute to 
(g — 2)^ at one loop, with a flavour-changing coupling at both vertices, and chirality-flip via a t mass insertion. 
One could hope to fit the (g — 2) M discrepancy, with a 2HDM containing the k t ^ coupling, but this would require 
cancellations in the generically more restrictive t4/j7 bound (see figure 4 and eqns (40), (45) and (48)). 

11 not considered in [3] 
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There are relevant contributions to r — > ^7 at one and two-loop; some diagrams are shown in figures 1 and 

2 

2. The one loop amplitude scales 12 as ^f- tan 2 /3 t (k ti1 tan/3 T ) 2 , and does not vanish for sin2(/3 — a) — > 0. The 
amplitude due to A exchange is of opposite sign from the h and H amplitudes, so the one-loop contribution to 
t->]U7 can be suppressed by tuning similar masses for h, H and A. Notice however, that the tuning required 
becomes finer with increasing tan/3 r , so at large tan/3 T , r decays are a more sensitive probe [5] of K T/J tan/3 T 
than hadron colliders. See [76] for a discussion of the promising prospects at an e — 7 collider. 

The two-loop "Barr-Zee" contributions to r — > fij can be of the same order as the one loop. The effective 
dipole interaction of r, \i and 7, must contain at least three Yukawa couplings at any loop order; if the Higgs has 
O(l) couplings to the tops, then two of the three Yukawa couplings in the two-loop diagram can be top Yukawas. 
At large tan/3 T , the </>- induced effective interaction ttf\i vanishes with sin2(/3 — a). In this sin2(/3 — a) — > 
limit, one of h and H becomes "Standard-Model-like" , with large couplings to the t and W, while A and the 
other CP-even scalar have unsuppressed flavour-changing interactions. However, as illustrated in figure 1, the 
"Barr-Zee" diagrams are closely related to Higgs production by gluon fusion. So if the contribution to r — > ^7 
is suppressed in this way, the collider cross-section is suppressed as well. The two loop top diagrams tend to 
dominate the r — > /17 amplitude at small tan/3 T and large sin2(/3 — a). Their contribution can be suppressed 
by cancellations between diagrams — for instance the h amplitude cancels against the H amplitude when 
1 / tan /3 T — > and m/j —¥ run — but these cancellations do not arise for the same parameter choices as suppress 
the one loop amplitude. 

So in practise, the model is described by m# + (taken ~ 300 GeV to satisfy b — > S7), rrih(— 114 GeV), vtia 
and itih (one of which should be "light" to avoid the decoupling limit, and the the T parameters constraint can 
be satisfied if the other is heavy) , tan (i T (<~ few, to maximise Higgs production by gluon fusion at the Tevatron 
and the LHC), and sin(/3 — a) (which is unconstrained if vtia — mn+, and is constrained to be ~ 1 by the T 
parameter if ran — m#+). Despite the freedom to vary sin(/3 — a), we find it difficult to generate a detectable 
cross-section at the Tevatron with parameters that satisfy the precision constraints. 

In the absence of cancellations, r — > fij is a more sensitive probe of the n Tfl coupling, than <j> — > t ± ^ =f at 
the TeVatron with <~ 5/fb data. This can be seen from figure 5. However, it is possible to evade constraints 
that arise from loops by tuning parameters, or by adding (more) New Physics. We have not studied this, for 
reasons of principle and practise. Our approach is phenomenological, so tuning masses and mixing angles to 
cancel one and two loop contributions appears ad hoc. In a model, cancellations can be justified by symmetries. 
More practically, we do not use complete gauge invariant two loop formulae, and do not have an estimate for 
higher order effects, so cancellations we find might not persist in a better analysis. 

Direct collider searches can give clearer bounds, being less dependent than loop processes on the vagaries of 
cancellations. Multiplying figure 5 by the current Tevatron bound on a(pp — > h$M — > t + t~) which is ~ 30<tsm, 
shows that <r(gg t ± A iT ) can be of order a(gg — > h$M — > t + t~), and consistent with r — > ^7 (for small 

tan/3 T ). So observing 4> — > r ± ^ =F in the near future would require "tuning" by the ratio plotted in figure 5: a 
factor ~ .1 for small tan/3 T . 
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